• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    _ 9 _ ABSTRACT OF THE DISCLOSURE A platform includes a head-mounted device, a multifunctional base, a screen and a computer system. A user wears the head-mounted device which is configured to provide augmented 5 reality content. The screen is disposed on the multifunctional base. The computer system is configured to provide video streaming to the screen. The screen rotates with respect to the user’s position. The user can experience augmented reality through this platform at arbitrary locations.
    公开号:NL2027188A
    申请号:NL2027188
    申请日:2020-12-21
    公开日:2021-08-23
    发明作者:Wang Chang-Song；Lai Yen-Ting；Chang Jen-Cheng；Su Kai-Hung；Chi Pai-Hung
    申请人:Adat Tech Co Ltd；
    IPC主号:
    专利说明:

    [01] [01] The present disclosure is related to an augmented reality (AR) and artificial intelligence (Al) integrated interactive display platform for users to experience both Al and AR.
    [02] [02] Augmented reality (AR) and artificial intelligence (Al) experience devices on the present market have only a limited range of movement, and thus users only experience AR at a fixed location. For example, if the AR and Al functions are designed for a huge machine which cannot be moved around, the users can only experience the AR and Al functions nearby the machine.SUMMARY
    [03] [03] Embodiments of the present disclosure provide an augmented reality and artificial intelligence integrated interactive display platform including a head-mounted device, a multifunctional base, a screen and a computer system. The head-mounted device is configured to provide augmented reality content. The multifunctional base includes a screen controlling unit. The screen is disposed on the multifunctional base and controlled by the screen controlling unit. The computer system is communicatively connected to the head- mounted device, the screen, and the multifunctional base, and configured to receive current position information from the head-mounted device, obtain a view angle of the head- mounted device according to the current position information, determine a content of a video streaming according to the view angle, and transmit the video streaming to the screen. The computer system is also configured to transmit a control signal to the screen controlling unit according to the view angle to rotate the screen such that a rotation angle of the screen is in response to the view angle.
    [04] [04] In some embodiments, the computer system is further configured to display a marker on the screen, and the head-mounted device includes an image sensor to capture an image of the marker. The computer system is configured to recognize the marker in the image to calculate a position of the head-mounted device relative to the screen.
    [05] [05] In some embodiments, which the position of the head-mounted device relative to the screen includes an initial vector. The current position information includes a three- dimensional coordinate of the head-mounted device, and the computer system is configured to calculate a difference between the three-dimensional coordinate of the head-mounted device and a three-dimensional coordinate of the screen to obtain a current vector, and 40 calculate the view angle according to the initial vector and the current vector.
    [06] [06] In some embodiments, the video streaming includes a 3D object, and the computer system is configured to rotate the 3D object according to the view angle.
    [07] [07] In some embodiments, the video streaming includes multiple frames of an object, and each of the frames corresponds to a different angle of the object. The computer system is configured to determine which of the frames to show on the screen according to the view angle.
    [08] [08] In some embodiments, the multifunctional base includes removable hollow pillars which are connected in series. One end of the removable hollow pillars is connected to the screen, and the other end of the removable hollow pillars is connected to the screen controlling unit. A power cable of the screen is disposed in the removable hollow pillars and is electrically connected to a power supply of the multifunctional base through a brush.
    [09] [09] In some embodiments, the multifunctional base further includes removable hollow protection shells which are connected in series and surround the removable hollow pillars. A height the removable hollow protection shells is lower than that of the removable hollow pillars.
    [10] [10] In some embodiments, the multifunctional base further includes a case, and a length of each of the removable hollow pillars and a length of each of the removable hollow protection shell are shorter than a length of the case.BRIEF DESCRIPTION OF THE DRAWINGS
    [11] [11] The invention can be more fully understood by reading the following detailed description of the embodiment, with reference made to the accompanying drawings as follows.
    [12] [12] FIG. 1 is a diagram illustrating an augmented reality and artificial intelligence integrated interactive display platform in accordance with some embodiments.
    [13] [13] FIG. 2 is a schematic diagram of the AIR integrated interactive display platform in accordance with some embodiments.
    [14] [14] FIG. 3 is a block diagram of the head-mounted device in accordance with some embodiments.
    [15] [15] FIG. 4A and FIG. 4B are diagrams of rotating the screen according to the view angle in accordance with some embodiments.
    [16] [16] FIG. 5 is a diagram illustrating frames of the video streaming having different angles in accordance with an embodiment.
    [17] [17] FIG. 6 is a front view of the AIR integrated interactive display platform in accordance with an embodiment.DETAILED DESCRIPTION
    [18] [18] Specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, however, the embodiments described 40 are not intended to limit the present invention and it is not intended for the description of
    [19] [19] The using of “first”, “second”, “third”, etc. in the specification should be understood for identifying units or data described by the same terminology, but are not referred to particular order or sequence.
    [20] [20] FIG. 1 is a diagram illustrating an augmented reality and artificial intelligence integrated interactive display platform in accordance with some embodiments. Referring to FIG. 1, the platform includes a head-mounted device 110, a screen 120, a computer system 130, and a multifunctional base 140. A user can wear the head-mounted device (HMD) 110 to experience augmented reality (AR) and artificial intelligence (Al) functions. Therefore, the computer system 130 may be referred to as an Al and AR (AIR) server.
    [21] [21] FIG. 2 is a schematic diagram of the AIR integrated interactive display platform in accordance with some embodiments. The multifunctional base 140 includes a screen controlling unit 141 (e.g. including a motor 141a and a motor controller 141b) and a communication module 142. The screen 120 may be a liquid crystal screen or an organic light emitting (OLED) screen. In some embodiments, the screen 120 is a touch screen. A wireless communication device of Wi-Fi is mounted on the screen 120, or the screen 120 supports Wi-Fi for receiving a video streaming from the computer system 130. The screen 120 is disposed on the multifunctional base 140 and controlled by the screen controlling unit 141 such that the screen 120 can rotate 360 degrees on the multifunctional base 140. The multifunctional base 140 is also electrically connected to the screen 120 for providing power. The computer system 130 is communicatively connected to the head-mounted device 110 and the screen 120 by wireless communication means such as Wi-Fi. In some embodiments, the communication module 142 is a Wi-Fi module for communicatively connecting to the computer system 130 and the head-mounted device 110. In some embodiments, the communication module 142 is a wire communication interface such as RS232 for communicatively connecting to the computer system 130.
    [22] [22] FIG. 3 is a block diagram of the head-mounted device in accordance with some embodiments. Referring to FIG. 3, the head-mounted device 110 includes a processor 310, a display 320, an image sensor 330, an inertial measurement unit (IMU) 340, a depth sensor 350, and a communication module 360. The processor 310 may be a central processing unit, a microcontroller, a microprocessor, or an application specific integrated circuit. The display 320 may be a transparent display for the user to see both of the real scene and virtual objects on the display 320. The image sensor 330 may include charge-coupled device (CCD) sensors, complementary metal-oxide semiconductor sensors, or other suitable light sensors. The inertial measurement unit 340 may include accelerometers, angular velocity sensors, magnetometers, etc. The depth sensor 350 may include an infrared emitter, an 40 infrared sensor, dual cameras, a structured light sensing device, or any suitable device
    [23] [23] The head-mounted device 110 provides augmented reality content which may be related to stand operation procedure (SOP), for example, to check or repair mechanical equipment. The head-mounted device 110 may also have navigation functions. For example, the head-mounted device 110 may perform simultaneous localization and mapping (SLAM) algorithms by the image sensor 330 and the depth sensor 350 or calculate direction and the location of itself by the inertial measurement unit 340.
    [24] [24] The video streaming shown on the screen 120 is, for example, related to a machine. When the user moves around, the screen 120 rotates correspondingly, and thus the user sees the screen 120 as a simulation machine to experience AR. In particular, the image sensor 330 captures images of the simulation machine to perform some Al modules such as object detection, object recognition. Accordingly, the user can also experience Al at the same time.
    [25] [25] FIG. 4A and FIG. 4B are diagrams of rotating the screen according to the view angle in accordance with some embodiments. In the initial stage, the computer system 130 shows a marker 410 on the screen 120. The marker 410 has particular shape, color, texture, or words. Prompt messages may be shown on the screen 120 to ask the user to look at the marker 410. Accordingly, the image sensor 330 of the head-mounted device 110 captures an image of the marker 410, and the head-mounted device 110 transmits the image to the computer system 130 which recognizes the marker in the image. The computer system 130 can calculate the position of the head-mounted device 110 relative to the computer system 130 because the size and the shape of the marker 410 on the screen 120 are predetermined. The relative position is represented as a vector Va. For example, the computer system 130 can calculate the distance between the screen 120 and the head-mounted device 110 according to the size of the marker in the image, and a view angle is set to be zero because the head-mounted device 110 should face toward the screen 120. In some embodiments, the computer system 130 can calibrate the view angle according to the shape variation of the marker in the image. In some embodiments, the vector Va can be used to calculate absolute coordinates. For example, the three-dimensional (3D) coordinates of the screen 120 can be set to be predetermined values, and the X coordinate and Y coordinate of the head-mounted device 120 can be calculated according to the vector Va and the 3D coordinates of the screen 120. The Z coordinate of the head-mounted device 120 can be calculated according to the height of the screen 120 and the shape variation of the marker in the image. In some embodiments, the head-mounted device 110 can calculate the 3D coordinates of itself based on SLAM technology, and add the 3D coordinates to the vector Va to obtain the 3D coordinates of the screen 120. In some embodiments, the recognition of the marker in the image is performed by the head-mounted device 110. In this case, the head-mounted device 110 calculates the distance between the screen 120 and the head- 40 mounted device 110 according to the size of the marker in the image, and the view angle is
    [26] [26] When the user moves, referring to FIG. 4B, the head-mounted device 110 transmits current position information of itself to the computer system 130. In some embodiments, the current position information includes 3D coordinates of the head-mounted device 110. The computer system 130 calculates the difference between the 3D coordinates of the head-mounted device 110 and the 3D coordinates of the screen 120 to obtain a current vector Vb. The computer system 130 calculates a view angle 8 according to the vector Va and the vector Vb. In some embodiments, the head-mounted device 110 calculates the view angle 8 which is included by the current position information. Herein, “the computer system 130 obtains the view angle according to the current position information” includes the aforementioned embodiments. Next, the computer system 130 transmits a control signal based on the view angle 6 to the screen controlling unit 141 to rotate the screen 120 such that the rotation angle of the screen 120 is in response to the view angle 9. For example, the screen 120 can rotate 6 degrees so that the user still faces toward the screen 120.
    [27] [27] In particular, the computer system 130 determines content of the video streaming according to the view angle 8, and transmits the video streaming to the screen
    [28] [28] FIG. 5 is a diagram illustrating frames of the video streaming having different angles in accordance with an embodiment. In the embodiments of FIG. 5, the aforementioned video streaming includes frames 510 of an object 520. Each frame 510 corresponds to different angles of the object 520. Note that the object 520 is merely for description, and the user only sees the frames 510 shown on the screen 120. The computer system 130 determines which of the frames 510 to show on the screen 120 according to the view angle 9. For example, there are N frames 510 where N is a positive integer. The computer system 130 can select (N x 8/360)" frame 510 to show on the screen 120. 40 [29] In some embodiments, the head-mounted device 110, the screen 120, the
    [30] [30] The disclosed platform provides users with a 360-degree AR experience of a real object with Al functions. At the same time, SOP simulation is also integrated into this platform. This platform introduces a 360-degree rotating screen for users to walk around the object.
    [31] [31] Although the present invention has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein. lt will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present invention without departing from the scope or spirit of the invention. In view of the foregoing, it is intended that the present invention cover modifications and variations of this invention provided they fall within the scope of the following claims.
    权利要求:
    Claims (8)
    [1]
    An augmented reality, augmented reality, and artificial intelligence, artificial intelligence, integrated interactive display, integrated interactive display, platform comprising: a head-mounted, head-mounted device configured to provide augmented reality content, content; a multifunctional base comprising a screen controller; a screen mounted on the multifunction base and controlled by the screen controller; and a computer system communicatively connected to the head-mounted device, the display, and the multifunction base, and configured to receive current position information from the head-mounted device, receive a viewing angle from the head-mounted device according to determine the current position information, determine a content of a video streaming according to the viewing angle, and transmit the video streaming to the screen, wherein the computer system is further configured to send a control signal to the screen controller according to the viewing angle to rotate the screen so that an angle of rotation of the screen is responsive to the viewing angle.
    [2]
    The augmented reality and artificial intelligence integrated interactive display platform of claim 1, wherein the computer system is further configured to display a marker on the screen, and the head-mounted device comprises an image sensor for capturing an image of the marker wherein the computer system is configured to recognize the marker in the image to calculate a position of the head-mounted device relative to the screen.
    [3]
    The augmented reality and artificial intelligence integrated interactive display platform of claim 2, wherein the position of the head-mounted device relative to the screen includes an initial vector, the current position information includes a three-dimensional coordinate of the head-mounted device, and the computer system is configured to calculate a difference between the three-dimensional coordinate of the head-mounted device and a three-dimensional coordinate of the screen to obtain current vector, and calculate the viewing angle corresponding to the initial vector and the current vector.
    [4]
    The augmented reality and artificial intelligence integrated interactive display platform of claim 3, wherein the video streaming comprises a 3D object, and the computer system rotates the 3D object according to the viewing angle.
    -8-
    [5]
    The augmented reality and artificial intelligence integrated interactive display platform of claim 3, wherein the video streaming includes a plurality of frames of an object, and each of the frames corresponds to a different angle of the object, wherein the computer system is configured to determine which of the frames to display on the screen according to the viewing angle.
    [6]
    The augmented reality and artificial intelligence integrated interactive display platform according to claim 3, wherein the multifunctional base comprises a plurality of removable hollow pillars connected in series, one end of the removable hollow pillars is connected to the display, and another end of the removable hollow pillars is connected to screen control unit, wherein a power cable of the screen is arranged in the removable hollow pillars and electrically connected to a power supply of the multifunctional base by a brush.
    [7]
    The augmented reality and artificial intelligence integrated interactive display platform according to claim 6, wherein the multifunctional base further comprises a plurality of removable hollow protection shells connected in series and surrounding the removable hollow pillars, and a height of the removable hollow protection shells is less than that of the removable hollow pillars.
    [8]
    The augmented reality and artificial intelligence integrated interactive display platform according to claim 7, wherein the multifunctional base further comprises a housing, and a length of each of the removable hollow pillars and a length of each of the removable hollow protection shells are less than a length of the housing.
    类似技术:
    公开号 | 公开日 | 专利标题
    NL2027188B1|2021-10-28|Augmented reality and artificial intelligence integrated interactive display platform
    US9804257B2|2017-10-31|Methods and systems for an immersive virtual reality system using multiple active markers
    KR20160148680A|2016-12-26|Determining coordinate frames in a dynamic environment
    US9858707B2|2018-01-02|3D video reconstruction system
    JP5869712B1|2016-02-24|Head-mounted display system and computer program for presenting a user's surrounding environment in an immersive virtual space
    EP3112986A1|2017-01-04|Content browsing
    CN104380347A|2015-02-25|Video processing device, video processing method, and video processing system
    EP2847616B1|2018-08-15|Surveying apparatus having a range camera
    US10474411B2|2019-11-12|System and method for alerting VR headset user to real-world objects
    US20210110141A1|2021-04-15|Methods and systems for training an object detection algorithm using synthetic images
    EP3128413A1|2017-02-08|Sharing mediated reality content
    EP3106963B1|2019-07-24|Mediated reality
    US10559131B2|2020-02-11|Mediated reality
    US11016559B2|2021-05-25|Display system and display control method of display system
    WO2017009529A1|2017-01-19|Mediated reality
    US9679352B2|2017-06-13|Method for operating a display device and system with a display device
    CN108021227B|2020-09-22|Method for rapidly moving in virtual reality and virtual reality device
    Kot et al.2017|Application of augmented reality in mobile robot teleoperation
    WO2018164287A1|2018-09-13|Method and device for providing augmented reality, and computer program
    KR20180062187A|2018-06-08|Method and apparatus for controlling displaying of augmented reality contents based on gyro sensor
    US9989762B2|2018-06-05|Optically composited augmented reality pedestal viewer
    US11269400B2|2022-03-08|Positioning system
    US10564801B2|2020-02-18|Method for communicating via virtual space and information processing apparatus for executing the method
    CN108446012B|2020-12-08|Virtual reality moving picture display method and virtual reality device
    EP3550405A1|2019-10-09|Cable detection for ar/vr computing method and apparatus
    同族专利:
    公开号 | 公开日
    GB2594121A|2021-10-20|
    NL2027188B1|2021-10-28|
    GB202020687D0|2021-02-10|
    TWM596351U|2020-06-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20070215776A1|2006-03-16|2007-09-20|Benq Corporation|Display panel and elevation adjusting base of the same|
    US20130314406A1|2012-05-23|2013-11-28|National Taiwan University|Method for creating a naked-eye 3d effect|
    US20180220110A1|2017-01-27|2018-08-02|Otoy, Inc.|Headphone based modular vr/ar platform with vapor display|
    CN109309873A|2017-07-28|2019-02-05|腾讯科技（深圳）有限公司|Screen angle method of adjustment, device and storage medium|
    US10809173B2|2017-12-15|2020-10-20|Analog Devices, Inc.|Smoke detector chamber boundary surfaces|
    USD920825S1|2018-11-06|2021-06-01|Analog Devices, Inc.|Smoke detector chamber|
    USD874964S1|2018-11-06|2020-02-11|Analog Devices, Inc.|Blocking members in a smoke detector chamber|
    US10921367B2|2019-03-06|2021-02-16|Analog Devices, Inc.|Stable measurement of sensors methods and systems|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    US201962955450P| true| 2019-12-31|2019-12-31|
    TW108217529U|TWM596351U|2019-12-31|2019-12-31|Augmented reality and artificial intelligence integrated interactive display platform|
    [返回顶部]